Crate serde_avro_fast
source ·Expand description
§Getting started
let schema: serde_avro_fast::Schema = r#"
{
"namespace": "test",
"type": "record",
"name": "Test",
"fields": [
{
"type": {
"type": "string"
},
"name": "field"
}
]
}
"#
.parse()
.expect("Failed to parse schema");
#[derive(serde_derive::Serialize, serde_derive::Deserialize, Debug, PartialEq)]
struct Test<'a> {
field: &'a str,
}
let rust_value = Test { field: "foo" };
let avro_datum = &[6, 102, 111, 111];
// Avro datum deserialization
assert_eq!(
serde_avro_fast::from_datum_slice::<Test>(avro_datum, &schema)
.expect("Failed to deserialize"),
rust_value
);
// Avro datum serialization
assert_eq!(
serde_avro_fast::to_datum(
&rust_value,
Vec::new(),
&mut serde_avro_fast::ser::SerializerConfig::new(&schema)
)
.expect("Failed to serialize"),
avro_datum
);
§Object container file encoding
Otherwise called “avro files”, avro object container files contain a header that holds the schema, followed by an arbitrary number of avro objects.
For this use-case, please see the object_container_file_encoding
module
documentation.
§Deriving schema from Rust structs
If the Rust program is the source of truth for the schema definition, it is
useful to define the schema as a derive on the relevant Rust structs.
This can be achieved using the serde_avro_derive
crate:
use serde_avro_derive::BuildSchema;
#[derive(BuildSchema)]
struct Foo {
primitives: Bar,
}
#[derive(BuildSchema)]
struct Bar {
a: i32,
b: String,
}
let schema: serde_avro_fast::Schema = Foo::schema()?;
// This will generate the following schema:
let _schema_str = r#"{
"type": "record",
"name": "crate_name.path.to.Foo",
"fields": [{
"name": "primitives",
"type": {
"type": "record",
"name": "Bar",
"fields": [
{ "name": "a", "type": "int" },
{ "name": "b", "type": "string" }
]
}
}]
}"#;
See the serde_avro_derive
documentation
for more details.
§An idiomatic (re)implementation of serde/avro (de)serialization
At the time of writing, the other existing libraries for Avro
(de)serialization do tons of unnecessary allocations, HashMap
lookups,
etc… for every record they encounter.
This version is a more idiomatic implementation, both with regards to Rust
and to serde
.
It is consequently >10x more performant (cf benchmarks):
apache_avro/small time: [386.57 ns 387.04 ns 387.52 ns]
serde_avro_fast/small time: [19.367 ns 19.388 ns 19.413 ns] <- x20 improvement
apache_avro/big time: [1.8618 µs 1.8652 µs 1.8701 µs]
serde_avro_fast/big time: [165.87 ns 166.92 ns 168.09 ns] <- x11 improvement
Re-exports§
pub use schema::Schema;
Modules§
- Defines everything necessary for avro deserialization
- Support for object container files
- Navigate, modify and initialize the
Schema
- Defines everything necessary for avro serialization
Functions§
- Deserialize from an avro “datum” (raw data, no headers…)
impl BufRead
- Deserialize from an avro “datum” (raw data, no headers…) slice
- Deserialize from an avro single object encoding
impl BufRead
- Deserialize from an avro single object encoding slice
- Serialize an avro “datum” (raw data, no headers…)
- Serialize an avro “datum” (raw data, no headers…)
- Serialize to an avro single object encoding
- Serialize to an avro single object encoding